Quality assessment, variability and reproducibility of anatomical measurements derived from T1-weighted brain imaging: The RIN-Neuroimaging Network case study

Initiatives for the collection of harmonized MRI datasets are growing continuously, opening questions on the reliability of results obtained in multi-site contexts. Here we present the assessment of the brain anatomical variability of MRI-derived measurements obtained from T1-weighted images, acquired according to the Standard Operating Procedures, promoted by the RINNeuroimaging Network. A multicentric dataset composed of 77 brain T1w acquisitions of young healthy volunteers (mean age = 29.7 +/- 5.0 years), collected in 15 sites with MRI scanners of three different vendors, was considered. Parallelly, a dataset of 7 "traveling" subjects, each undergoing three acquisitions with scanners from different vendors, was also used. Intra-site, intra-vendor, and inter-site variabilities were evaluated in terms of the percentage standard deviation of volumetric and cortical thickness measures. Image quality metrics such as contrast-to-noise and signal-to-noise ratio in gray and white matter were also assessed for all sites and vendors. The results showed a measured global variability that ranges from 11% to 19% for subcortical volumes and from 3% to 10% for cortical thicknesses. Univariate distributions of the normalized volumes of subcortical regions, as well as the distributions of the thickness of cortical parcels appeared to be significantly different among sites in 8 subcortical (out of 17) and 21 cortical (out of 68) regions of i nterest in the multicentric study. The Bland-Altman analysis on "traveling" brain measurements did not detect systematic scanner biases even though a multivariate classification approach was able to classify the scanner vendor from brain measures with an accuracy of 0.60 +/- 0.14 (chance level 0.33).